#9 (reply in thread)
#9 (reply in thread)

Key Features Implemented:

• Triplet Tracking with Triplet struct tracking parent segment, nextOffset address, and actual segment for proper relationship management between nested containers used for automatic promotion to extended containers

• BFS/DFS Style Element Access with NextSegment method for horizontal BFS traversal, PushSegment and PopSegment methods for nested DFS traversal, and GetBytes providing BFS access across all segments

• Automatic Promotion to Extended Containers where nested containers automatically become extended when size exceeds pivotSize and large data automatically creates extended containers with proper header updates

• Continuation Addresses with 32-bit offsets for extended containers lifting the 13-bit limit, proper chain linking with SelfOffset and Continuation fields, and EndOfChain marker for final segment

• Simplified API with single ExtendedContainer type replacing ExtendedPutAccess and ExtendedGetAccess, consistent naming like AddInt16 and AddString instead of AddInt16Extended, and ExtendedReader providing unified reading interface

Technical Details:

• ExtendedContainer struct contains segments slice, triplets slice, current PutAccess pointer, pivotSize int threshold, isExtended bool flag, parent ExtendedContainer pointer, and parentOffsetAddr int

• Triplet struct contains ParentSegment byte slice, NextOffsetAddr int, ActualSegment byte slice, IsExtended bool, SelfOffset uint32, and Continuation uint32

• ExtendedReader struct contains segments 2D byte slice, currentSeg int index, segmentStack slice of ExtendedReader pointers, and getAccess GetAccess pointer

Include buffer pooling via existing PutAccess and GetAccess sync.Pool, zero-copy operations where possible, automatic segmentation optimizing memory usage, and extended containers adding 8-byte header overhead per segment.

@heartical bro I merged it, but it will not propogated to main with the current.
could you redo it the way I asked. firstly we should write encoding side. then decode and so on.

@heartical
for an example we choose to omit the first chunk inside
this should be a simple test for string (for now as demo it will think 16bytes string as big).

type ExtendedContainerValue struct {
    NextSegmentOffset uint32
   //for now no additional info or self check, we can check always size of extendedContainer and decide if it contains chunk or not
}

func TestExtendedContainerPack(t *testing.T) {
    put := NewExtendedPutAccess(16)
    put.AddInt16(42)
    put.AddBool(true)
    put.AddString(strings.Repeat("A", 16)) // 16‑byte string triggers extended container
    put.AddBytes([]byte{0xAA, 0xBB})

    actual, err := put.PackExtended()
    require.NoError(t, err)

    // --- First segment: headers + primitive payload + ExtendedContainer marker ---
    expectedFirstSegment := []byte{
        0x51, 0x00, // TypeInt16
        0x15, 0x00, // TypeBool
        0x22, 0x00, // ExtendedContainer
        0x2E, 0x00, // TypeBytes
        0x38, 0x00, // TypeEnd

        0x2A, 0x00,             // int16(42)
        0x01,                   // bool(true)
        0x00, 0x00, 0x00, 0x0D, // nextDataOffset
        0xAA, 0xBB,             // bytes payload
    }

    if !bytes.Equal(actual[:len(expectedFirstSegment)], expectedFirstSegment) {
        t.Errorf("First segment mismatch.\nExpected: %v\nActual:   %v",
            expectedFirstSegment, actual[:len(expectedFirstSegment)])
    }

    // --- Second segment: ExtendedContainer with chunk ---
    // Expected layout:
    // 22 00 (ExtendedContainer)
    //  C0 00   // TypeEnd (offset 24)
    // 00 00 00 00 (NextSegmentOffset = 0, last segment)
    // 26 00 (TypeString)
    // 80 00 (TypeEnd)
    // 16 bytes of 'A'
    expectedSecondSegment := []byte{
        0x22, 0x00, // ExtendedContainer  length will be 24 and we will know that it has chunk inside
        0xC0, 0x00, // TypeEnd  // 
        0x00, 0x00, 0x00, 0x00, // NextSegmentOffset
        0x26, 0x00, // TypeString
        0x80, 0x00, // TypeEnd
        // 16 bytes of 'A'
        'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
        'A', 'A', 'A', 'A', 'A', 'A', 'A', 'A',
    }

    segStart := len(expectedFirstSegment)
    if !bytes.Equal(actual[segStart:segStart+len(expectedSecondSegment)], expectedSecondSegment) {
        t.Errorf("Second segment mismatch.\nExpected: %v\nActual:   %v",
            expectedSecondSegment, actual[segStart:segStart+len(expectedSecondSegment)])
    }
}

then tuple, map, and at the end root itself.

• you can store segments inside parent PutExtendedAccess
• triplets could use ID instead of *[]byte.
• triplets storage could be tree making concatentaion bfs way as well
• in the string case above if its 14kb it should be in 2-3 (1st just empty placeholder like above) 2nd segment (8192-10) , and last one in the next

adr02 was rebased onto main. and also you were added as a contributor.
adr001 needs to be redone gradual way. starting with encoding. the current one does not comply with it fully. thats why I will keep its codes inside experimental. check discussions.

thanks a lot.